It is often desirable to create panoramic images from a sequence of photographs. For example, a user may capture a series of two dimensional (2D) images of a scene on a mobile phone or handheld camera, with some overlap between images, by moving or panning the camera in a particular direction. The individual images may then be integrated or stitched together to create the panorama. Similarly, frames from a video sequence that pan over a scene may be subsequently stitched together. Most 2D stitching techniques assume that the scene is sufficiently far from the camera and/or does not contain many depth layers that can generate parallax effects (i.e., an apparent difference in position of an object when viewed from varying angles).
When three dimensional (3D) cameras are used, however, most of the objects of interest in the scene tend to be close to the camera and the scenes in general include objects at multiple depths, making it difficult to stitch frames together. The images provided by 3D cameras are generally composed of pixels providing both color information (e.g., red, green, blue or RGB data) and depth information. Some methods of stitching together 3D images attempt to incorporate the depth information to construct 3D geometric models of parts of the scene, but this often yields poor color and texture rendition. These methods also suffer from the need to collect relatively large quantities of image data over time. Other methods, such as stereoscopic stitching typically produce unacceptable distortion of object shapes when a scene has multiple or complex depth layers.
An additional problem that is encountered when trying to integrate multiple images into a panoramic scene using existing techniques is the inability to deal with dynamic foreground objects against a static background. Moving objects can change shape and position from one image frame to the next, resulting in visually undesirable and confusing results after integration.
Although the following Detailed Description will proceed with reference being made to illustrative embodiments, many alternatives, modifications, and variations thereof will be apparent to those skilled in the art.